Analyzing Concepts Over Time

ABSTRACT

A method and apparatus are provided for automatically generating and processing first and second concept vector sets extracted, respectively, from a first set of concept sequences and from a second, temporally separated, concept sequences by performing a natural language processing (NLP) analysis of the first concept vector set and second concept vector set to detect changes in the corpus over time by identifying changes for one or more concepts included in the first and/or second set of concept sequences.

BACKGROUND OF THE INVENTION

In the field of artificially intelligent computer systems capable ofanswering questions posed in natural language, cognitive questionanswering (QA) systems (such as the IBM Watson™ artificially intelligentcomputer system or and other natural language question answeringsystems) process questions posed in natural language to determineanswers and associated confidence scores based on knowledge acquired bythe QA system. In operation, users submit one or more questions througha front-end application user interface (UI) or application programminginterface (API) to the QA system where the questions are processed togenerate answers that are returned to the user(s). The QA systemgenerates answers from an ingested knowledge base corpus, includingpublicly available information and/or proprietary information stored onone or more servers, Internet forums, message boards, or other onlinediscussion sites. Using the ingested information, the QA system canformulate answers using artificial intelligence (AI) and naturallanguage processing (NLP) techniques to provide answers with associatedevidence and confidence measures. However, the quality of the answerdepends on the ability of the QA system to identify and processinformation contained in the knowledge base corpus.

With some traditional QA systems, there are mechanisms provided forprocessing information in a knowledge base by using vectors to representwords to provide a distributed representation of the words in alanguage. Such mechanisms include “brute force” learning by varioustypes of Neural Networks (NNs), learning by log-linear classifiers, orvarious matrix formulations. Lately, word2vec, that uses classifiers,has gained prominence as a machine learning technique which is used inthe natural language processing and machine translation domains toproduce vectors which capture syntactic as well semantic properties ofwords. Matrix based techniques that first extract a matrix from the textand then optimize a function over the matrix have recently achievedsimilar functionality to that of word2vec in producing vectors. However,there is no mechanism in place to identify and/or process concepts in aningested corpus which are more than merely a sequence of words. Nor aretraditional QA systems able to identify and process concept attributesin relation to other concept attributes. Instead, existing attempts todeal with concepts generate vector representations of words that carryvarious probability distributions derived from simple text in a corpus,and therefore provide only limited capabilities for applications, suchas NLP parsing, identification of analogies, and machine translation. Asa result, the existing solutions for efficiently identifying andapplying concepts contained in a corpus are extremely difficult at apractical level.

SUMMARY

Broadly speaking, selected embodiments of the present disclosure providea system, method, and apparatus for processing of inquiries to aninformation handling system capable of answering questions by using thecognitive power of the information handling system to generate orextract a sequence of concepts, to extract or compute therefrom adistributed representation of the concept(s) (i.e., concept vectors),and to process the distributed representation (the concept vectors) tocarry out useful tasks in the domain of concepts and user-conceptinteraction. In selected embodiments, the information handling systemmay be embodied as a question answering (QA) system which has access tostructured, semi-structured, and/or unstructured content contained orstored in one or more large knowledge databases (each a.k.a., “corpus”),and which extracts therefrom a sequence of concepts from annotated text(e.g., hypertext with concept links highlighted), from graphrepresentations of concepts and their inter-relations, from tracking thenavigation behavior of users, or a combination thereof. In otherembodiments, concept vectors may also be used in a “discovery advisor”context where users would be interested in seeing directly theconcept-concept relations, and/or use query concepts to retrieve andrelate relevant documents from a corpus. To compute the conceptvector(s), the QA system may process statistics of associations in theconcept sequences using vector embedding methods. Processing of thegenerated concept vectors and their similarities over time enablesimproved presentation and visualization of concepts and theirinter-relations to improve the quality of answers provided by the QAsystem. For example, changes in a corpus over time can be detected bycomparing two states (e.g., old and new) of a concept graph a certaintime period apart from one another (e.g., two years apart) to identifythe significant changes that occurred in relationship strengths betweenconcepts, thereby enabling identification of trends. In addition, thecomparison of two states of the concept graph a certain time periodapart from one another may be used to identify new concepts that havenewly appeared with strong relationships to concepts that are central toa technology area of interest, thereby enabling identification ofdisruptive concepts.

The foregoing is a summary and thus contains, by necessity,simplifications, generalizations, and omissions of detail; consequently,those skilled in the art will appreciate that the summary isillustrative only and is not intended to be in any way limiting. Otheraspects, inventive features, and advantages of the present invention, asdefined solely by the claims, will become apparent in the non-limitingdetailed description set forth below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention may be better understood, and its numerousobjects, features, and advantages made apparent to those skilled in theart by referencing the accompanying drawings, wherein:

FIG. 1 depicts a network environment that includes a knowledge managerthat uses a knowledge base and a vector concept engine for identifyingand processing vector concepts extracted from the knowledge base;

FIG. 2 is a block diagram of a processor and components of aninformation handling system such as those shown in FIG. 1;

FIG. 3 illustrates a simplified flow chart showing the logic forobtaining and using a distributed representation of concepts as vectors;

FIG. 4 graphically illustrates an example of how the relationshipbetween first and second concept sequences changes over time;

FIG. 5 illustrates a simplified flow chart showing the logic for using atemporal comparison of two states of a concept graph to identify changesin relationship strengths between the involved concepts;

FIG. 6 illustrates a simplified flow chart showing the logic for using atemporal comparison of two states of a concept graph to identify newdisruptive concepts that have newly appeared with strong relationshipsto concepts that are central to a technology area of interest; and

FIG. 7 illustrates a simplified flow chart showing the logic forchecking for the appearance of emerging concepts with strong affinity toa specific field.

DETAILED DESCRIPTION

The present invention may be a system, a method, and/or a computerprogram product. In addition, selected aspects of the present inventionmay take the form of an entirely hardware embodiment, an entirelysoftware embodiment (including firmware, resident software, micro-code,etc.) or an embodiment combining software and/or hardware aspects thatmay all generally be referred to herein as a “circuit,” “module” or“system.” Furthermore, aspects of the present invention may take theform of computer program product embodied in a computer readable storagemedium (or media) having computer readable program instructions thereonfor causing a processor to carry out aspects of the present invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a dynamic or static random access memory(RAM), a read-only memory (ROM), an erasable programmable read-onlymemory (EPROM or Flash memory), a magnetic storage device, a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Java, Smalltalk, C++ or the like,and conventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server or cluster of servers. Inthe latter scenario, the remote computer may be connected to the user'scomputer through any type of network, including a local area network(LAN) or a wide area network (WAN), or the connection may be made to anexternal computer (for example, through the Internet using an InternetService Provider). In some embodiments, electronic circuitry including,for example, programmable logic circuitry, field-programmable gatearrays (FPGA), or programmable logic arrays (PLA) may execute thecomputer readable program instructions by utilizing state information ofthe computer readable program instructions to personalize the electroniccircuitry, in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

FIG. 1 depicts a schematic diagram of one illustrative embodiment of aquestion/answer (QA) system 100 connected to a computer network 102. TheQA system 100 may include one or more QA system pipelines 100A, 100B,each of which includes a knowledge manager computing device 104(comprising one or more processors and one or more memories, andpotentially any other computing device elements generally known in theart including buses, storage devices, communication interfaces, and thelike) for processing questions received over the network 102 from one ormore users at computing devices (e.g., 110, 120, 130). Over the network102, the computing devices communicate with each other and with otherdevices or components via one or more wired and/or wireless datacommunication links, where each communication link may comprise one ormore of wires, routers, switches, transmitters, receivers, or the like.In this networked arrangement, the QA system 100 and network 102 mayenable question/answer (QA) generation functionality for one or morecontent users. Other embodiments of QA system 100 may be used withcomponents, systems, sub-systems, and/or devices other than those thatare depicted herein.

In the QA system 100, the knowledge manager 104 may be configured toreceive inputs from various sources. For example, knowledge manager 104may receive input from the network 102, one or more knowledge bases orcorpora of electronic documents 106 which stores electronic documents107, semantic data 108, or other possible sources of data input. Inselected embodiments, the knowledge database 106 may include structured,semi-structured, and/or unstructured content in a plurality of documentsthat are contained in one or more large knowledge databases or corpora.The various computing devices (e.g., 110, 120, 130) on the network 102may include access points for content creators and content users. Someof the computing devices may include devices for a database storing thecorpus of data as the body of information used by the knowledge manager104 to generate answers to questions. The network 102 may include localnetwork connections and remote connections in various embodiments, suchthat knowledge manager 104 may operate in environments of any size,including local and global, e.g., the Internet. Additionally, knowledgemanager 104 serves as a front-end system that can make available avariety of knowledge extracted from or represented in documents,network-accessible sources and/or structured data sources. In thismanner, some processes populate the knowledge manager, with theknowledge manager also including input interfaces to receive knowledgerequests and respond accordingly.

In one embodiment, the content creator creates content in electronicdocuments 107 for use as part of a corpus of data with knowledge manager104. Content may also be created and hosted as information in one ormore external sources 16-18, whether stored as part of the knowledgedatabase 106 or separately from the QA system 100A. Wherever stored, thecontent may include any file, text, article, or source of data (e.g.,scholarly articles, dictionary definitions, encyclopedia references, andthe like) for use in knowledge manager 104. Content users may accessknowledge manager 104 via a network connection or an Internet connectionto the network 102, and may input questions to knowledge manager 104that may be answered by the content in the corpus of data. As furtherdescribed below, when a process evaluates a given section of a documentfor semantic content 108, the process can use a variety of conventionsto query it from the knowledge manager. One convention is to send aquestion 10. Semantic content is content based on the relation betweensignifiers, such as words, phrases, signs, and symbols, and what theystand for, their denotation, or connotation. In other words, semanticcontent is content that interprets an expression, such as by usingNatural Language (NL) Processing. In one embodiment, the process sendswell-formed questions 10 (e.g., natural language questions, etc.) to theknowledge manager 104. Knowledge manager 104 may interpret the questionand provide a response to the content user containing one or moreanswers 20 to the question 10. In some embodiments, knowledge manager104 may provide a response to users in a ranked list of answers 20.

In some illustrative embodiments, QA system 100 may be the IBM Watson™QA system available from International Business Machines Corporation ofArmonk, N.Y., which is augmented with the mechanisms of the illustrativeembodiments described hereafter for identifying and processing conceptvectors which may aid in the process of answering questions. The IBMWatson™ knowledge manager system may receive an input question 10 whichit then parses to extract the major features of the question, that inturn are used to formulate queries that are applied to the corpus ofdata stored in the knowledge base 106. Based on the application of thequeries to the corpus of data, a set of hypotheses, or candidate answersto the input question, are generated by looking across the corpus ofdata for portions of the corpus of data that have some potential forcontaining a valuable response to the input question.

In particular, a received question 10 may be processed by the IBMWatson™ QA system 100 which performs deep analysis on the language ofthe input question 10 and the language used in each of the portions ofthe corpus of data found during the application of the queries using avariety of reasoning algorithms. There may be hundreds or even thousandsof reasoning algorithms applied, each of which performs differentanalysis, e.g., comparisons, and generates a score. For example, somereasoning algorithms may look at the matching of terms and synonymswithin the language of the input question and the found portions of thecorpus of data. Other reasoning algorithms may look at temporal orspatial features in the language, while others may evaluate the sourceof the portion of the corpus of data and evaluate its veracity.

The scores obtained from the various reasoning algorithms indicate theextent to which the potential response is inferred by the input questionbased on the specific area of focus of that reasoning algorithm. Eachresulting score is then weighted against a statistical model. Thestatistical model captures how well the reasoning algorithm performed atestablishing the inference between two similar passages for a particulardomain during the training period of the IBM Watson™ QA system. Thestatistical model may then be used to summarize a level of confidencethat the IBM Watson™ QA system has regarding the evidence that thepotential response, i.e., candidate answer, is inferred by the question.This process may be repeated for each of the candidate answers until theIBM Watson™ QA system identifies candidate answers that surface as beingsignificantly stronger than others and thus, generates a final answer,or ranked set of answers, for the input question. The QA system 100 thengenerates an output response or answer 20 with the final answer andassociated confidence and supporting evidence. More information aboutthe IBM Watson™ QA system may be obtained, for example, from the IBMCorporation website, IBM Redbooks, and the like. For example,information about the IBM Watson™ QA system can be found in Yuan et al.,“Watson and Healthcare,” IBM developerWorks, 2011 and “The Era ofCognitive Systems: An Inside Look at IBM Watson and How it Works” by RobHigh, IBM Redbooks, 2012.

To improve the quality of answers provided by the QA system 100, theconcept vector engine 13 may be embodied as part of a QA informationhandling system 15 in the knowledge manager 104, or as a separateinformation handling system, to execute a concept vector identificationprocess that extracts a sequence of concepts from annotated text sources16 (e.g., sources specializing in concepts, such as Wikipedia pages withconcepts highlighted or hyperlinked), from graph representations 17 ofconcepts and their inter-relations, from tracking the navigationbehavior of users 18, or a combination thereof, and to constructtherefrom one or more vectors for each concept 110. Syntactically, a“concept” is a single word or a word sequence (e.g., “gravity”, “supremecourt”, “Newton's second law”, “Albert Einstein”) which becomes asemantic “concept” once it has been designated by a community to have aspecial role, namely—as representing more than just a sequence of words.In addition, a concept has many attributes: field of endeavor, origin,history, an associated body of work and/or knowledge, cultural and/orhistorical connotation and more. So, although superficially, words,phrases and concepts seem similar, a word sequence becomes a conceptwhen it embeds a wider cultural context and a designation by acommunity, encompassing a significant meaning and presence in an area,in a historical context, in its relationships to other concepts and inways it influences events and perceptions. It is worth emphasizing thepoint that not every well-known sequence of words is a concept, and thedeclaration of a sequence of words to be a concept is a communitydecision which has implications regarding naturally-arising sequences ofconcepts. With this understanding, the concept vector engine 13 mayinclude a concept sequence identifier 11 which accesses sources 16-18for sequences of concepts embedded in texts of various kinds and/orwhich arise by tracking concept exploration behavior from examiningnon-text sources, such as click streams. As different concept sequencesare identified, the adjacency of the concepts is tied to the closenessof the concepts themselves. Once concept sequences are available, aconcept vector extractor 12 acts as a learning device to extract vectorrepresentations for the identified concepts. The resulting conceptvectors 110 may be stored in the knowledge database 106 or directlyaccessed by one or more vector processing applications 14 such as, forexample, analyzing concepts over time to identify market trends ordisruptive technologies.

To identify or otherwise obtain a sequence of concepts, a concept vectoridentifier 11 may be provided to (i) access one or more wiki pages 16 orother text source which contains these concepts by filtering out wordsthat are not concepts, (ii) algorithmically derive concept sequencesfrom a graph 17 (e.g., a Concept Graph (CG)), (iii) track one or moreactual users' navigation behavior 18 over concepts, or some modificationor combination of one of the foregoing. For example, the concept vectoridentifier 11 may be configured to extract concepts from a text source,but also some text words extracted per concept in the contextsurrounding the concept's textual description, in which case theconcepts are “converted” to new unique words.

To provide a first illustrative example, the concept vector identifier11 may be configured to derive concept sequences 11A from one or moreWikipedia pages 16 by eliminating all words from a page that are notconcepts (i.e., Wikipedia entries). For example, consider the followingsnippet from the Wikipedia page for Photonics athttp://en.wikipedia.org/wiki/Photoics in which the concepts areunderlined:

-   -   Photonics as a field began with the invention of the laser        in 1960. Other developments followed: the laser diode in the        1970s, optical fibers for transmitting information, and the        erbium-doped fiber amplifier. These inventions formed the basis        for the telecommunications revolution of the late 20th century        and provided the infrastructure for the Internet.    -   Though coined earlier, the term photonics came into common use        in the 1980s as fiber-optic data transmission was adopted by        telecommunications network operators. At that time, the term was        used widely at Bell Laboratories. Its use was confirmed when the        IEEE Lasers and Electro-Optics Society established an archival        journal named Photonics Technology Letters at the end of the        1980s.    -   During the period leading up to the dot-com crash circa 2001,        photonics as a field focused largely on optical        telecommunications.

In this example, the concept sequence 11A derived by the concept vectoridentifier 11 is: laser, laser diode, optical fibers, erbium-doped fiberamplifier, Internet, Bell Laboratories, IEEE Lasers and Electro-OpticsSociety, Photonics Technology Letters, dot-com crash. However, it willbe appreciated that the concept vector identifier 11 may examine a“dump” of Wikipedia pages 16 to obtain long concept sequences reflectingthe whole collection of Wikipedia concepts.

In another illustrative example, the concept vector identifier 11 may beconfigured to derive concept sequences 11A from one or more specificdomains. For example, a pharmaceutical company's collection of concerneddiseases, treatments, drugs, laboratory tests, clinical trials, relevantchemical structures and processes, or even biological pathways may beaccessed by the concept vector identifier 11 to extract domain-specificconcept sequences. In this example, concept sequences may be extractedfrom company manuals, emails, publications, reports, and othercompany-related text sources.

In another illustrative example, the concept vector identifier 11 may beconfigured to derive concept sequences 11A which also includenon-concept text. For example, an identified concept sequence mayinclude inserted “ordinary” or non-concept words which are used forlearning. One option would be to use all the words from the originalsource text by converting “concept” words into “new” words by appendinga predetermined suffix (e.g., “_01”) to each concept. In the example“Photonics” page listed above, this approach would lead to the followingfirst paragraph: “Photonics as a field began with the invention of thelaser 01 in 1960. Other developments followed: the laser diode 01 in the1970s, optical fibers 01 for transmitting information, and theerbium-doped fiber amplifier 01. These inventions formed the basis forthe telecommunications revolution of the late 20th century and providedthe infrastructure for the Internet 01.”

Another option for deriving concept sequences with text would be toprocess the original source text by a filtering process that retainsonly the parts of the text relevant to a specific theme. For example, ifthe original source text consists of a collection of medical documents,a search procedure can be applied to identify and retrieve only thedocuments containing the word “cancer.” The retrieved documents aretaken as the theme-restricted collection for deriving the conceptsequences.

Another option for deriving concept sequences with text would be toprocess the original source text to keep only words that are somewhatinfrequent as indicated by an occurrence threshold, and that are inclose proximity to a concept. In the example “Photonics” page listedabove, this approach would lead to the following first paragraph:“invention laser 01 1960. developments laser diode 01 1970s, opticalfibers 01 transmitting information erbium-doped fiber amplifier 01telecommunications revolution infrastructure Internet 01.”

Another option for deriving concept sequences is to construct sequencesof concepts and words in units and (potentially rearranged) orderings,as determined by a natural language parser.

Another option for deriving concept sequences with text would be toexplicitly specify a collection of words or types of words to beretained in the concept sequence. For example, one may have a specifiedcollection of words connected to medicine (e.g., nurse, doctor, ward andoperation), and the derived concept sequence would limit retainednon-concept words or text to this specified collection.

To provide a second illustrative example of the concept sequenceidentifier process, the concept vector identifier 11 may be configuredto derive concept sequences (e.g., 11A) from one or more concept graphs17 having nodes which represent concepts (e.g., Wikipedia concepts). Aswill be appreciated, a graph 17 may be constructed by any desired method(e.g., Google, etc.) to define “concept” nodes which may be tagged withweights indicating their relative importance. In addition, an edge ofthe graph is labeled with the strength of the connection between theconcept nodes it connects. When edge weights are given, they indicatethe strength or closeness of these concepts, or observed and recordedvisits by users in temporal proximity. An example way of relating theedge weights to user visits is to define the edge weight connectingconcept “A” to concept “B” to be the number of times users examinedconcept “A” and, within a short time window, examined concept “B”.

Using the Wikipedia example, if a Wikipedia page “A” has a link toanother Wikipedia page “B,” then the graph 17 would include an edgeconnecting the “A” concept to the “B” concept. The weight of a node(importance) or the weight (strength) of an edge of an edge may bederived using any desired technique, such as a personalized Pagerank ofthe graph or other techniques. In addition, each concept i in the graph17 may be associated with a (high dimensional) P-vector such that thej^(th) entry of the P-vector corresponding to concept i is the strengthof the connection between concept i and concept j. The entries of theP-vector may be used to assign weights to graph edges. To derive conceptsequences from the concept graph(s) 17, the concept vector identifier 11may be configured to perform random walks on the concept graph(s) 17 andview these walks as concept sequences. For example, starting with arandomly chosen starting node v, the concept vector identifier 11examines the G-neighbors of v and the weights on the edges connecting vand its neighboring nodes. Based on the available weights (if none areavailable, the weights are considered to be equal), the next node israndomly chosen to identify the next node (concept) in the sequencewhere the probability to proceed to a node depends on the edge weightand the neighboring node's weight relative to other edges andneighboring nodes. This random walk process may be continued until aconcept sequence of length His obtained, where H may be a specifiedparametric value (e.g., 10,000). Then, the random walk process may berepeated with a new randomly selected starting point. If desired, theprobability of selecting a node as a starting node may be proportionalto its weight (when available). The result of a plurality of randomwalks on the graph 17 is a collection of length H sequences of concepts11A.

Extracting sequences from the concept graph(s) 17 may also be done byusing a random walk process in which each step has a specifiedprobability that the sequence jumps back to the starting concept node(a.k.a., “teleportation”), thereby mimicking typical navigationbehavior. Alternatively, a random walk process may be used in which eachstep has a specified probability that the sequence jumps back to theprevious concept node, thereby mimicking other typical navigationbehavior. If desired, a combination of the foregoing step sequences maybe used to derive a concept sequence. Alternatively, a concept sequencemay be derived by using a specified user behavior model M thatdetermines the next concept to explore. Such a model M may employ a moreelaborate scheme in order to determine to which concept a user willexamine next, based on when previous concepts were examined and for whatduration.

The resulting concept sequences 11A may be stored in the knowledgedatabase 109 or directly accessed by the concept vector extractor 12. Inaddition, whenever changes are made to a concept graph 17, the foregoingprocess may be repeated to dynamically maintain concept sequences byadding new concept sequences 11A and/or removing obsolete ones. Byrevisiting the changed concept graph 17, previously identified conceptsequences can be replaced with new concept sequences that would havebeen used, thereby providing a controlled time travel effect.

In addition to extracting concepts from annotated text 16 and/or graphrepresentations 17, concept sequences 11A may be derived usinggraph-based vector techniques whereby an identified concept sequence 11Aalso includes a vector representation of the concept in the context ofgraph G (e.g., Pagerank-derived vectors). This added information aboutthe concepts in the sequence 11A can be used to expedite andqualitatively improve the learning of parameters process, and learningquality, by providing grouping, i.e., additional information aboutconcepts and their vicinity as embedded in these G-associated vectors.

To provide a third illustrative example of the concept sequenceidentifier process, the concept vector identifier 11 may be configuredto derive concept sequences (e.g., 11A) from the user navigationbehavior 18 where selected pages visited by a user (or group of users)represent concepts. For example, the sequences of concepts may be theWikipedia set of entries explored in succession by (a) a particularuser, or (b) a collection of users. The definition of succession mayallow non-Wikipedia intervening web exploration either limited byduration T (before resuming Wikipedia), number of interveningnon-Wikipedia explorations, or a combination of theses or relatedcriteria. As will be appreciated, user navigation behavior 18 may becaptured and recorded using any desired method for tracking a sequenceof web pages a user visits to capture or retain the “concepts”corresponding to each visited page and to ignore or disregard the pagesthat do not correspond to concepts. Each concept sequence 11A derivedfrom the captured navigation behavior 18 may correspond to a particularuser, and may be concatenated or combined with other user's conceptsequences to obtain a long concept sequence for use with concept vectortraining. In other embodiments, the navigation behavior of a collectionof users may be tracked to temporally record a concept sequence from allusers. While such collective tracking blurs the distinction betweenindividual users, this provides a mechanism for exposing a group effort.For example, if the group is a limited-size departmental unit (say, upto 20), the resulting group sequence 11A can reveal interestingrelationships between the concepts captured from the user navigationbehavior 18. The underlying assumption is that the group of users isworking on an interrelated set of topics.

To provide another illustrative example of the concept sequenceidentifier process, the concept vector identifier 11 may be configuredto generate concept sequences using concept annotations created by twoor more different annotators, where each annotator uses its chosen setof names to refer to the collection of concepts included in a textsource. For example, one annotator applied to a text source may mark upall occurrences of the concept of “The United State of America” as“U.S.A.”, whereas another may mark it up as “The United States”. Inoperation, a first concept sequence may be generated by extracting afirst plurality of concepts from a first set of concept annotations forthe one or more content sources, and a second concept sequence may begenerated by extracting a second plurality of concepts from a second setof concept annotations for the one or more content sources. In this way,the concept vector identifier 11 may be used to bring together differentannotated versions of a corpus. In another example, a first set ofconcept annotations may be a large collection of medical papers that aremarked up with concepts that are represented in the Unified MedicalLanguage System (UMLS) Metathesaurus. The second set of conceptannotations may the same collection of medical papers that are marked upwith concepts that are defined in the English Wikipedia. Since these twodictionaries have good overlap but they are not identical, they mayrefer to the same thing (e.g. leukemia) differently in the differentsets of concept annotations.

In addition to identifying concept sequences 11A from one or moreexternal sources 16-18, general concept sequences may be constructed outof extracted concept sequences. For example, previously captured conceptsequences 109 may include a plurality of concept sequences S1, S2, . . ., Sm which originate from various sources. Using these conceptsequences, the concept vector identifier 11 may be configured to form along sequence S by concatenating the sequences S=S1S2 . . . Sm.

Once concept sequences 11A are available (or stored 109), a conceptvector extractor 12 may be configured to extract concept vectors 12Abased on the collected concept sequences. For example, the conceptvector extractor 12 may employ a vector embedding system (e.g.,Neural-Network-based, matrix-based, log-linear classifier-based or thelike) to compute a distributed representation (vectors) of concepts 12Afrom the statistics of associations embedded within the conceptsequences 11A. More generally, the concept vector extractor 12 embodiesa machine learning component which may use Natural Language Processingor other techniques to receive concept sequences as input. Thesesequences may be scanned repeatedly to generate a vector representationfor each concept in the sequence by using a method, such as word2vec.Alternatively, a matrix may be derived from these sequences and afunction is optimized over this matrix and word vectors, and possiblycontext vectors, resulting in a vector representation for each conceptin the sequence. Other vector generating methods, such as using NeuralNetworks presented by a sequence of examples derived from the sequences,are possible. The resulting concept vector may be a low dimension (about100-300) representation for the concept which can be used to compute thesemantic and/or grammatical closeness of concepts, to test for analogies(e.g., “a king to a man is like a queen to what?”) and to serve asfeatures in classifiers or other predictive models. The resultingconcept vectors 12A may be stored in the knowledge database 110 ordirectly accessed by one or more vector processing applications 14.

To generate concept vectors 12A, the concept vector extractor 12 mayprocess semantic information or statistical properties deduced from wordvectors extracted from the one or more external sources 16-18. To thisend, the captured concept sequences 11A may be directed to the conceptvector extraction function or module 12 which may use Natural LanguageProcessing (NLP) or machine learning processes to analyze the conceptsequences 11A to construct one or more concept vectors 12A, where “NLP”refers to the field of computer science, artificial intelligence, andlinguistics concerned with the interactions between computers and human(natural) languages. In this context, NLP is related to the area ofhuman-to-computer interaction and natural language understanding bycomputer systems that enable computer systems to derive meaning fromhuman or natural language input. To process the concept sequences 11A,the concept vector extractor 12 may include a learning or optimizationcomponent which receives concept sequence examples 11A as Neural Networkexamples, via scanning text, and the like. In the learning component,parameters (Neural Network weights, matrix entries, coefficients insupport vector machines (SVMs), etc.) are adjusted to optimize a desiredgoal, usually reducing an error or other specified quantity. Forexample, the learning task in the concept vector extractor 12 may beconfigured to implement a scanning method where learning takes place bypresenting examples from a very large corpus of Natural Language (NL)sentences. The examples may be presented as Neural Network examples, inwhich the text is transformed into a sequence of examples where eachexample is encoded in a way convenient for the Neural Network intake, orvia scanning text where a window of text is handled as a word sequencewith no further encoding. In scanning methods, the learning task isusually to predict the next concept in a sequence, the middle concept ina sequence, concepts in the context looked at as a “bag of words,” orother similar tasks. The learning task in the concept vector extractor12 may be also configured to implement a matrix method wherein textcharacteristics are extracted into a matrix form and an optimizationmethod is utilized to minimize a function expressing desired word vectorrepresentation. The learning results in a matrix (weights, parameters)from which one can extract concept vectors, or directly in conceptvectors (one, or two per concept), where each vector Vi is associatedwith a corresponding concept Ci. Once the learning task is complete, theproduced concept vectors may have other usages such as measuring“closeness” of concepts (usually in terms of cosine distance) or solvinganalogy problems of the form “a to b is like c to what?”

To provide a first illustrative example for computing concept vectorsfrom concept sequences, the concept vector extractor 12 may beconfigured to employ vector embedding techniques (e.g., NN,matrix-based, log-linear classifier or the like) whereby “windows” of k(e.g., 5-10) consecutive concepts are presented and one is “taken out”as the concept to be predicted. The result is a vector representationfor each concept. Alternatively, the concept vector extractor 12 may beconfigured to use a concept to predict its neighboring concepts, and thetraining result produces the vectors. As will be appreciated, othervector producing methods may be used. Another interesting learning taskby which vectors may be created is that of predicting the next fewconcepts or the previous few concepts (one sided windows).

To provide another illustrative example for computing concept vectors12A from concept sequences 11A, the concept vector extractor 12 may beconfigured to employ NLP processing techniques to extract a distributedrepresentation of NLP words and obtain vectors for the conceptidentifiers. As will be appreciated, the size of the window may belarger than those used in the NLP applications so as to allow forconcepts to appear together in the window. In addition, a filter F whichcan be applied to retain non-concept words effectively restricts thewords to only the ones that have a strong affinity to their nearbyconcepts as measured (for example, by their cosine distance to theconcept viewed as a phrase in an NLP word vector production, e.g., byusing word2vec).

To provide another illustrative example for computing concept vectors12A from concept sequences 11A, the concept vector extractor 12 may beconfigured to employ NLP processing techniques to generate differentconcept vectors from different concept sequences by supplying a firstplurality of concepts (extracted from a first set of conceptannotations) as input to the vector learning component to generate thefirst concept vector and by supplying a second plurality of concepts(extracted from a second set of concept annotations) as input to thevector learning component to generate a second concept vector. If bothversions of concept sequence annotations are brought together to obtainfirst and second concept vectors, the resulting vectors generated fromthe different concept sequence annotations can be compared to oneanother by computing similarities therebetween. As will be appreciated,different annotators do not always mark up the same text spans inexactly the same way, and when different annotation algorithms choose tomark up different occurrences of the term, a direct comparison of theresulting concept vectors just by text alignment techniques is nottrivial. However, if both versions of annotated text sources areincluded in the embedding process, by way of association with otherconcepts and non-concept words, the respective concept vectors can bebrought to close proximity in the embedding space. Computingsimilarities between the vectors could reveal the linkage between suchalternative annotations.

Once concept vectors 12A are available (or stored 110), they can bemanipulated in order to answer questions such as “a king is to man islike a queen is to what?”, cluster similar words based on a similaritymeasure (e.g., cosine distance), or use these vectors in otheranalytical models such as a classification/regression model for makingvarious predictions. In addition, one or more vector processingapplications 14 may be applied to carry out useful tasks in the domainof concepts and user-concept interaction, allowing better presentationand visualization of concepts and their inter-relations (e.g.,hierarchical presentation, grouping, and for a richer and more efficientuser navigation over the concept graph). In processing concept vectors,an application 14 may access n vectors V1, . . . , Vn of dimension dwhich represent n corresponding concepts C1, . . . , Cn, where a vectorVi is a tuple (vil, . . . , vid) of entries where each entry is a realnumber. Concept vector processing may include the computation of the dotproduct of two vectors Vh and Vi, denoted dot(Vh, Vi) is Σj=1, . . . , dVhj*Vij. In concept vectors processing, the length of vector Vi isdefined as the square root of dot(Vi, Vi), i.e., SQRT(dot(Vi, Vi)). Inaddition, the cosine distance between Vh and Vi, denoted cos(Vh, Vi), isdot(Vh, Vi)/(length(Vh)*length(Vi)). The cosine distance is a measure ofsimilarity, where a value of “1” indicates very high similarity and avalue of “−1” indicates very weak similarity. As will be appreciated,there are other measures of similarity that may be used to processconcept vectors, such as soft cosine similarity.

To provide a first illustrative example application for processingconcept vectors 12A, a vector processing application 14 may beconfigured to detect changes in a corpus or corpora over time bycomparing temporally separated concepts. The characterization of corpuschanges could involve global summaries of the set of concepts appearingin the corpus over different time points, analyses of the relationshipbetween concepts that persist over time, and the detection of appearanceof new concepts or disappearance of old concepts. Each of these gives away to highlight the changes in contents of the corpus, which may revealmarket trends, emergence of new technologies or new social-politicalissues, and other differences. For example, a user can compare twostates of a concept collection, or collections, over time (e.g., 2 yearsapart) and ask for major changes in relationship strengths between theinvolved concepts, thereby enabling detection of market trends and othertemporal changes. In response, the concept vector engine 13 and/orvector processing application 14 may compare two concept sequences S1and S2 taken at two collection snapshots in time, where S1 containsconcepts C1, . . . , Ck, and where S2 contains C1, . . . , Ck and “new”concepts N1, . . . , Nb; there is no limitation here in that, if neededS2 may be concatenated with a concept sequence derived from S1 to ensurethat C1, . . . , Ck appear in S2. In addition, the vector processingapplication 14 may be configured to compute or learn concept vectorrepresentations V1, . . . , Vk and V1′, . . . , V′k, V′k+1, . . . V′k+bover S1 and S2, respectively. To identify the concepts whoseinterrelationship has significantly changed, the vector processingapplication 14 may be configured to compute cos(Vi, Vj), cos(V′i, V′j)for all i≥1 that is different from j and less than k+1, and then reportpairs such that |cos(Vi, Vj)−cos(V′i, V′j)| exceeds a first reportingthreshold, DELTA1 (a parameter, for example 0.3). In addition or in thealternative, the “new” concepts with a strong relationship to “old”concepts may be identified by configuring the vector processingapplication 14 to compute cos(V′i, V′j) for 0<i<k+1 and k<j<b+1, andreport pairs such that cos(V′i, V′j) exceeds a second, larger reportingthreshold, DELTA2 (a parameter, for example 0.8).

To provide another illustrative example application for processingconcept vectors 12A, a vector processing application 14 may beconfigured to identify disruptive concepts, such as new concepts thatmay represent disruptive technology (e.g., not well known) with strongaffinity to concepts that are central to a technology in which the useris interested. For example, a user who is exploring concepts (e.g.,Wikipedia concepts) in a specified technology area may request thatdisruptive concepts be identified. In response, the concept vectorengine 13 and/or vector processing application 14 may be configured toidentify new concepts N1, . . . , Nb and compute their correspondingvectors V′k+1, . . . , V′k+b. In addition, a list of concepts L=(L1, . .. , Lh) are identified that are central to a technology in which a useris interested, and the corresponding concept vectors VL1, . . . , VLhare extracted. With the identified concepts and vectors, the vectorprocessing application 14 may be configured to sort the new concepts N1,. . . , Nb by their highest cosine distance to any of VL1, . . . , VLh,and then report the top R new concepts as being disruptive concepts,where R is a programmable parameter (e.g., R=3).

Types of information handling systems that can use the QA system 100range from small handheld devices, such as handheld computer/mobiletelephone 110 to large mainframe systems, such as mainframe computer170. Examples of handheld computer 110 include personal digitalassistants (PDAs), personal entertainment devices, such as MP3 players,portable televisions, and compact disc players. Other examples ofinformation handling systems include a pen or tablet computer 120,laptop or notebook computer 130, personal computer system 150, andserver 160. As shown, the various information handling systems can benetworked together using computer network 102. Types of computer network102 that can be used to interconnect the various information handlingsystems include Local Area Networks (LANs), Wireless Local Area Networks(WLANs), the Internet, the Public Switched Telephone Network (PSTN),other wireless networks, and any other network topology that can be usedto interconnect the information handling systems. Many of theinformation handling systems include nonvolatile data stores, such ashard drives and/or nonvolatile memory. Some of the information handlingsystems may use separate nonvolatile data stores (e.g., server 160utilizes nonvolatile data store 165, and mainframe computer 170 utilizesnonvolatile data store 175). The nonvolatile data store can be acomponent that is external to the various information handling systemsor can be internal to one of the information handling systems.

FIG. 2 illustrates an illustrative example of an information handlingsystem 200, more particularly, a processor and common components, whichis a simplified example of a computer system capable of performing thecomputing operations described herein. Information handling system 200includes one or more processors 210 coupled to processor interface bus212. Processor interface bus 212 connects processors 210 to Northbridge215, which is also known as the Memory Controller Hub (MCH). Northbridge215 connects to system memory 220 and provides a means for processor(s)210 to access the system memory. In the system memory 220, a variety ofprograms may be stored in one or more memory device, including a conceptvector application engine module 221 which may be invoked to processuser interactions and data sources to identify concept sequences andextract therefrom concept vectors which may be used in variousapplications, such as analyzing concepts over time to identify trends,disruptive concepts and/or emerging concepts. Graphics controller 225also connects to Northbridge 215. In one embodiment, PCI Express bus 218connects Northbridge 215 to graphics controller 225. Graphics controller225 connects to display device 230, such as a computer monitor.

Northbridge 215 and Southbridge 235 connect to each other using bus 219.In one embodiment, the bus is a Direct Media Interface (DMI) bus thattransfers data at high speeds in each direction between Northbridge 215and Southbridge 235. In another embodiment, a Peripheral ComponentInterconnect (PCI) bus connects the Northbridge and the Southbridge.Southbridge 235, also known as the I/O Controller Hub (ICH) is a chipthat generally implements capabilities that operate at slower speedsthan the capabilities provided by the Northbridge. Southbridge 235typically provides various busses used to connect various components.These busses include, for example, PCI and PCI Express busses, an ISAbus, a System Management Bus (SMBus or SMB), and/or a Low Pin Count(LPC) bus. The LPC bus often connects low-bandwidth devices, such asboot ROM 296 and “legacy” I/O devices (using a “super I/O” chip). The“legacy” I/O devices (298) can include, for example, serial and parallelports, keyboard, mouse, and/or a floppy disk controller. Othercomponents often included in Southbridge 235 include a Direct MemoryAccess (DMA) controller, a Programmable Interrupt Controller (PIC), anda storage device controller, which connects Southbridge 235 tononvolatile storage device 285, such as a hard disk drive, using bus284.

ExpressCard 255 is a slot that connects hot-pluggable devices to theinformation handling system. ExpressCard 255 supports both PCI Expressand USB connectivity as it connects to Southbridge 235 using both theUniversal Serial Bus (USB) and the PCI Express bus. Southbridge 235includes USB Controller 240 that provides USB connectivity to devicesthat connect to the USB. These devices include webcam (camera) 250,infrared (IR) receiver 248, keyboard and trackpad 244, and Bluetoothdevice 246, which provides for wireless personal area networks (PANs).USB Controller 240 also provides USB connectivity to other miscellaneousUSB connected devices 242, such as a mouse, removable nonvolatilestorage device 245, modems, network cards, ISDN connectors, fax,printers, USB hubs, and many other types of USB connected devices. Whileremovable nonvolatile storage device 245 is shown as a USB-connecteddevice, removable nonvolatile storage device 245 could be connectedusing a different interface, such as a Firewire interface, etc.

Wireless Local Area Network (LAN) device 275 connects to Southbridge 235via the PCI or PCI Express bus 272. LAN device 275 typically implementsone of the IEEE 802.11 standards for over-the-air modulation techniquesto wireless communicate between information handling system 200 andanother computer system or device. Extensible Firmware Interface (EFI)manager 280 connects to Southbridge 235 via Serial Peripheral Interface(SPI) bus 278 and is used to interface between an operating system andplatform firmware. Optical storage device 290 connects to Southbridge235 using Serial ATA (SATA) bus 288. Serial ATA adapters and devicescommunicate over a high-speed serial link. The Serial ATA bus alsoconnects Southbridge 235 to other forms of storage devices, such as harddisk drives. Audio circuitry 260, such as a sound card, connects toSouthbridge 235 via bus 258. Audio circuitry 260 also providesfunctionality such as audio line-in and optical digital audio in port262, optical digital output and headphone jack 264, internal speakers266, and internal microphone 268. Ethernet controller 270 connects toSouthbridge 235 using a bus, such as the PCI or PCI Express bus.Ethernet controller 270 connects information handling system 200 to acomputer network, such as a Local Area Network (LAN), the Internet, andother public and private computer networks.

While FIG. 2 shows one example configuration for an information handlingsystem 200, an information handling system may take many forms, some ofwhich are shown in FIG. 1. For example, an information handling systemmay take the form of a desktop, server, portable, laptop, notebook, orother form factor computer or data processing system. In addition, aninformation handling system may take other form factors such as apersonal digital assistant (PDA), a gaming device, ATM machine, aportable telephone device, a communication device or other devices thatinclude a processor and memory. In addition, an information handlingsystem need not necessarily embody the north bridge/south bridgecontroller architecture, as it will be appreciated that otherarchitectures may also be employed.

To provide additional details for an improved understanding of selectedembodiments of the present disclosure, reference is now made to FIG. 3which depicts a simplified flow chart 300 showing the logic forobtaining and using a distributed representation of concepts as vectors.The processing shown in FIG. 3 may be performed in whole or in part by acognitive system, such as the QA information handing system 15, QAsystem 100, or other natural language question answering system whichidentifies sequences of concepts to extract concept vectors (e.g.,distributed representations of the concept) which may be processed tocarry out useful tasks in the domain of concepts and user-conceptinteraction.

FIG. 3 processing commences at 301 whereupon, at step 302, a question orinquiry from one or more end users is processed to generate an answerwith associated evidence and confidence measures for the end user(s),and the resulting question and answer interactions are stored in aninteraction history database. The processing at step 302 may beperformed at the QA system 100 or other NLP question answering system,though any desired information processing system for processingquestions and answers may be used. As described herein, a NaturalLanguage Processing (NLP) routine may be used to process the receivedquestions and/or generate a computed answer with associated evidence andconfidence measures. In this context, NLP is related to the area ofhuman-computer interaction and natural language understanding bycomputer systems that enable computer systems to derive meaning fromhuman or natural language input.

In the course of processing questions to generate answers, a collectionor sequence of concepts may be processed at step 310. The conceptsequence processing at step 310 may be performed at the QA system 100 orconcept vector engine 13 by employing NLP processing and/or extractionalgorithms, machine learning techniques, and/or manual processing tocollect concepts from one or more external sources (such as theWikipedia or some other restricted domain, one or more concept graphsources, and/or captured user navigation behavior) to generate traininginput comprising concept sequences. As will be appreciated, one or moreprocessing steps may be employed to obtain the concept sequences.

For example, the concept sequence processing at step 310 may employ oneor more concept graphs to generate concept sequences at step 303. Tothis end, the concept graph derivation step 303 may construct a graph Gusing any desired technique (e.g., a graph consisting of Wikipediaarticles as nodes and the links between them as edges) to defineconcepts at each graph node which may be tagged with weights indicatingits relative importance. In addition, the graph edges may be weighted toindicate concept proximity. By traversing the graph G using theindicated weights to affect the probability of navigating via an edge, asequence of concepts may be constructed at step 303. In contrast toexisting approaches for performing short random walks on graph nodeswhich view these as sentences and extract a vector representation foreach node, the graph derivation step 303 may employ a random walk thatis directed by the edge weights such that there is a higher probabilityto traverse heavier weight edges, thereby indicating closeness ofconcepts. In addition, the concept graphs employed by the graphderivation step 303 encodes many distinct domains may be represented asgraphs that are derived non-trivially from the conventional web graph.In addition, the graph derivation step 303 may allow a graph traversalwith a “one step back” that is not conventionally available. As aresult, the resulting concept vectors are quite different.

In addition or in the alternative, the concept sequence processing atstep 310 may employ one or more text sources to extract conceptsequences at step 304. In selected embodiments, the text source is theWikipedia set of entries or some other restricted domain. By analyzing alarge corpus of documents mentioning Wikipedia entries (e.g., Wikipediaitself and other documents mentioning its entries), the text sourceextraction step 304 may extract the sequence of concepts, including thetitle, but ignoring all other text. In addition, the text sourceextraction step 304 may extract the sequence of appearing concepts alongwith additional words that are extracted with the concept in the contextof surrounding its textual description while using a filter to removeother words not related to the extracted concepts. Alternatively, thetext source extraction step 304 may extract a mixture of concepts andtext by parsing a text source to identify concepts contained therein,replacing all concept occurrences with unique concept identifiers (e.g.,by appending a suffix to each concept or associating critical words withconcepts).

In addition or in the alternative, the concept sequence processing atstep 310 may employ behavior tracking to derive concept sequences atstep 305. In selected embodiments, the actual user's navigation behavioris tracked to use the actual sequence of explored concepts by a singleuser or a collection of users to derive the concept sequence at step305. In selected embodiments, the tracking of user navigation behaviormay allow non-Wikipedia intervening web exploration that is limited byduration T before resuming Wikipedia, by the number of interveningnon-Wikipedia explorations, by elapsed time or a combination of these orrelated criteria.

After the concept sequence processing step 310, the collected conceptsequences may be processed to compute concept vectors using known vectorembedding methods at step 311. As disclosed herein, the concept vectorcomputation processing at step 311 may be performed at the QA system 100or concept vector extractor 12 by employing machine learning techniquesand/or NLP techniques to compute a distributed representation (vectors)of concepts from the statistics of associations. As will be appreciated,one or more processing steps may be employed to compute the conceptvectors. For example, the concept vector computation processing at step311 may employ NL processing technique such as word2vec or to implementa neural network (NN) method at step 306 to perform “brute force”learning from training examples derived from concept sequences providedby step 310. In addition or in the alternative, the concept vectorcomputation processing at step 311 may employ various matrixformulations at method step 307 and/or extended with SVM-based methodsat step 308. In each case, the vector computation process may use alearning component in which selected parameters (e.g., NN weights,matrix entries, vector entries, etc.) are repeatedly adjusted until adesired level of learning is achieved.

After the concept vector computation processing step 311, the computedconcept vectors may be used in various applications at step 312 whichmay be performed at the QA system 100 or the concept vector applicationmodule 14 by employing NLP processing, artificial intelligence,extraction algorithms, machine learning model processing, and/or manualprocessing to process the distributed representation (concept vectors)to carry out useful tasks in the domain of concepts and user-conceptinteraction. For example, a temporal concept analysis application 309performed at step 312 may compare states of a concept graph at twodifferent, separate times to identify significant changes that occurredin relationship strengths between concepts, thereby allowing for anunderstanding of trends, such as market trends. In addition or in thealternative, the temporal concept analysis application 309 performed atstep 312 may compare two states of the concept graph a certain timeperiod apart and provide new concepts that have appeared with strongrelationships to concepts that are central to a technology area ofinterest, thereby allowing for an understanding of disruptivetechnologies. As will be appreciated, each of the concept vectorapplications 309 executed at step 312 can be tailored or constrained toa specified domain by restricting the corpus input to only documentsrelevant to the domain and/or restricting concept sequences to thedomain and/or restricting remaining words to those of significance tothe domain.

To illustrate further details of selected embodiments of the temporalconcept analysis application described herein, reference is now made toFIG. 4 which graphically illustrates the chronological relationshipbetween a first concept sequence S1 401 and a second concept sequence S2402 to illustrate how the concept sequences S1, S2 change over time. Thetwo depicted concept sequences S1 401 and S2 402 are taken at twosnapshots in time such that the S2 402 snapshot time is after the S1 401snapshot time. As a result, the first concept sequence S1 401 is the“older” sequence which includes “old” concepts {a, b, c, d, e} in thesequence, ab c d b a b c d e a a b . . . b. In addition, the secondconcept sequence S2 402 is the “newer” sequence which includes the “old”concepts {a, b, c, d, e} and the “new” concepts {m, n, q} in thesequence, m m a b q m a n a n q m b . . . b. Stated more generally, theold concept sequence S1 401 contains the concept set {C1, . . . , Ck},and the new concept sequence S2 402 contains the concept set {C1, . . ., Ck} and the “new” concept set {N1, . . . , Nb}.

In accordance with the present disclosure, the old and new conceptsequences S1 401, S2 402 may be processed to detect changes in a corpusover time since the contents of a corpus at any particular time aresummarized by the concepts extracted from it at that time. As a result,changes in concept distributions between old and new concept sequencesS1 401, S2 402 will signify changes in the corpus. To detect suchchanges, vector representation of the concepts can be used to describethe corpus contents at a particular time by the shape and connectivityof the region(s) spanned by the concept vectors. And by comparing theconcept vectors from different times, changes in the corpus betweendifferent time points can be identified by recognizing changes in thevalues of quantitative features that characterize the geometry andtopology of the concept regions. These may include positions of thecentroids, diameters between extreme points, orientations of principalaxes, number of significant dimensions, aspect ratios, and quantitiesthat can be computed by many types of clustering algorithms (e.g.k-means or agglomerative clustering). These could identify shifts inemphases, differences in themes, and concentration and dissolution oftopic groups.

To provide additional details for an improved understanding of selectedembodiments of the present disclosure, reference is now made to FIG. 5which depicts a simplified flow chart 500 showing the logic for using atemporal comparison of two states of a concept collection to identifychanges in relationship strengths between the involved concepts. Theprocessing shown in FIG. 5 may be performed in whole or in part by acognitive system, such as the QA information handing system 15, QAsystem 100, or other natural language question answering system whichidentifies and compares temporally separated sequences of concepts toidentify changes in relationship strengths between involved concepts tocarry out useful tasks in the domain of concepts and user-conceptinteraction, such as identifying market trends.

FIG. 5 processing commences at step 501 by capturing, retrieving, orotherwise obtaining at least two temporally separated input sequences,including an “old” sequence S1 and a “new” sequence S2, taken atsnapshots spaced apart by a certain amount of time (e.g., 2 years). Inselected embodiments, the input sequences S1, S2 may be retrieved fromstorage in a database, where S1 contains the concept set {C1, . . . ,Ck}, and where S2 contains the “old” set {C1, . . . , Ck} and the “new”concept set {N1, . . . , Nb}. The processing at step 501 may beperformed at the QA system 100 or other NLP question answering system,though any desired information processing system for processingquestions and answers may be used.

Once the concept sequences S1, S2 are available, the concept sequencesmay be processed to compute or train concept vectors V1, . . . , Vk, forthe concepts in the first input sequence S1 using known vector embeddingtechniques, performed on S1, at step 502. As disclosed herein, theconcept vector computation processing at step 502 may be performed atthe QA system 100 or concept vector extractor 12 by employing machinelearning techniques and/or NLP techniques to compute a distributedrepresentation (vectors) of concepts V1, . . . , Vk which are trained onthe concepts from the first input sequence S1. For example, the conceptvector computation processing at step 502 may employ NL processingtechnique such as word2vec or to implement a neural network (NN) methodto perform “brute force” learning from training examples derived fromthe S 1 concept sequences. In addition or in the alternative, theconcept vector computation processing at step 502 may employ variousmatrix formulations and/or extended with SVM-based methods. In eachcase, the vector computation process may use a learning component inwhich selected parameters (e.g., NN weights, matrix entries, vectorentries, etc.) are repeatedly adjusted until a desired level of learningis achieved.

At step 503, the concept sequences S1, S2 are further processed tocompute or train concept vectors V′1, . . . , V′k, V′k+1, . . . V′k+bover S1S2, the concatenation of S1 and S2. Here, Vi and Vi′, 1≤i≤krefers to Ci, while V′i, i>k, refers to Ni−k. In other words, 1 . . . kare indices of concepts in both S1 and S2, and k+1 . . . , k+b areindices of concepts only in S2. As disclosed herein, the concept vectorcomputation processing at step 503 may be performed at the QA system 100or concept vector extractor 12 by employing known vector embeddingtechniques, such as machine learning and/or NLP techniques, to compute adistributed representation (vectors) of concepts V′1, . . . , V′k,V′k+1, . . . V′k+b which are trained on the concepts from theconcatenation S1S2.

At step 504, the concept sequences S1, S2 are further processed todetect changes in similarities between persistent concepts. As disclosedherein, the concept vector processing at step 504 may be performed atthe QA system 100 or vector processing applications 14 by computingcos(Vi, Vj), cos(V′i, V′j) for all i that is different from j for 1≤i,j≤k, and then reporting pairs such that |cos(Vi, Vj)−cos(V′i, V′j)|exceeds a first reporting threshold, DELTA1 (a parameter, for example0.2). Intuitively, these reported pairs are “old” concepts whoseinterrelationship has significantly changed. In this way, the processingstep 504 may identify significant changes in similarities orinterrelationships between the persistent concepts that are common tothe two, temporally separated versions of the corpus.

In an alternative embodiment for detecting changes in similaritiesbetween persistent concepts as set forth in steps 503, 504, the conceptsequences S1, S2 may be processed at step 506 to compute or trainconcept vectors V′1, . . . , V′k, V′k+1, . . . V′k+b over the newconcept sequence S2. Here, Vi and Vi′, 1≤i≤k refers to Ci and V′i, i>k,refers to Ni−k. In other words, 1 . . . k are concepts in both S1 andS2, and k+1 . . . , k+b are concepts only in S2. In the computations ofstep 506, there is no concatenation, and the training at steps 502 and506 simply address snapshots at two different points in time. Thistraining is appropriate when the snapshot time of S1 is significantlydifferent than the snapshot time of S2. As disclosed herein, the conceptvector computation processing at step 506 may be performed at the QAsystem 100 or concept vector extractor 12 by employing known vectorembedding techniques, such as machine learning and/or NLP techniques, tocompute a distributed representation (vectors) of concepts V′1, . . . ,V′k, V′k+1, . . . V′k+b which are trained on the concepts from the newconcept sequence S2.

At step 507, the concept sequences S1, S2 are further processed at theQA system 100 or vector processing applications 14 by computing cos(Vi,Vj), cos(V′i, V′j) for all i that is different from j, with i,j≤k, andthen reporting pairs such that |cos(Vi, Vj)−cos(V′i, V′j)| exceeds asecond reporting threshold, DELTA2 (a parameter, for example 0.25).Intuitively, these reported pairs are “old” concepts whoseinterrelationship has significantly changed. Usually, the reportingthreshold DELTA2 is larger than DELTA1 as the “past” (inclusion of S1)“dampens” the vector differences in the training over S1S2 and sousually DELTA1<DELTA2. In this way, the processing step 507 may detectold concepts whose relationship had significantly changed.

Building on the results of processing step 504, the concept sequencesS1, S2 may be processed further at step 505 to detect new concepts inthe later corpus that are strongly similar to old concepts in theearlier corpus. As disclosed herein, the concept vector processing atstep 505 may be performed at the QA system 100 or vector processingapplications 14 by using the vectors V′1, . . . , V′k, V′k+1, . . .V′k+b learned at step 503 from the concatenated concept sequence S1S2(i.e., the set of both old and new concepts). As a first step, thevector processing application 14 may be configured to compute cos(V′i,V′j) for 1≤i≤k and k<j≤k+b. In addition, the vector processingapplication 14 may be configured to report pairs such that cos(V′i, V′j)exceeds a third reporting threshold, DELTA3 (a parameter, for example0.5). Intuitively, these are “new” concepts with a strong relationshipto “old” concepts.

To provide additional details for an improved understanding of selectedembodiments of the present disclosure, reference is now made to FIG. 6which depicts a simplified flow chart 601-604 showing the logic forusing a temporal comparison of two concept vector states to identify newdisruptive concepts that have newly appeared with strong relationshipsto concepts that are central to a technology area of interest. Theprocessing shown in FIG. 6 may be performed in whole or in part by acognitive system, such as the QA information handing system 15, QAsystem 100, or other natural language question answering system whichidentifies and compares temporally separated sequences of concepts tocarry out useful tasks in the domain of concepts and user-conceptinteraction, such as identifying disruptive technologies with strongaffinity to concepts that are central to a technology area of interest.

FIG. 6 processing commences at step 601 by capturing, retrieving, orotherwise obtaining at least two temporally separated input sequences,including an “old” sequence S1 and a “new” sequence S2 spaced apart by acertain amount of time. In selected embodiments, the input sequences S1,S2 may be retrieved from storage in a database, where S1 contains theconcept set {C1, . . . , Ck}, and where S2 contains the “old” set {C1, .. . , Ck} and the “new” concept set {N1, . . . , Nb}. The processing atstep 601 may be performed at the QA system 100 or other NLP questionanswering system, though any desired information processing system forprocessing questions and answers may be used. Once the concept sequencesS1, S2 are available, the concept sequences may be processed to computeor train concept vectors V1, . . . , Vk, for the concepts in the firstinput sequence S1 using known vector embedding techniques at step 601.In addition, the concept sequences S1, S2 are further processed tocompute or train concept vectors V′1, . . . , V′k, V′k+1, . . . V′k+bover S1S2, the concatenation of S1 and S2. At step 601, Vi and Vi′,1≤i≤k refers to concepts in both S1 and S2, while Vi, i>k (e.g., i=k+1 .. . k+b) refer to concepts only in S2.

At step 602, the concept sequences S1, S2 are further processed,manually by human experts or via specific tools, to identify conceptsL=L1, . . . Lh which are central to a specified technology area T. Usingthe identified concepts L, the corresponding concept vectors VL1, . . ., VLh are also identified which are obtained by training over S1S2. Asdisclosed herein, the processing at step 602 may be performed at the QAsystem 100 or concept vector extractor 12 by employing known vectorembedding techniques, such as machine learning and/or NLP techniques, tocompute a distributed representation (vectors) of concepts VL1, . . . ,VLh which are trained on the concepts from the concatenation S1S2.

At step 603, the vectors V′k+1, . . . , V′k+b for concepts N1, . . . ,Nb are sorted to identify disruptive concepts. As disclosed herein, thevector sorting at step 603 may be performed at the QA system 100 orconcept processing application 14 by employing any desired sortingtechnique to sort the computed cosine distance from high to low, such asby sorting the vectors V′k+1, . . . , V′k+b by their highest cosinedistance to any of VL1, . . . , VLh. The concept processing step 603 maythen generate a sorted vector list LS along with a list LSC of thecorresponding concepts, and then report the top R (new) concepts fromthe list of the corresponding concepts LSC, where R is a parameter(e.g., 3). In this way, the reported R concepts are potentially emergingdisruptive technologies or potentially disruptive legal, economic,political or business concepts.

At step 604, the vectors V′k+1, . . . , V′k+b for concepts N1, . . . ,Nb are further sorted and processed to identify disruptive concepts thatrelate “holistically” to a specified technology area T that may berepresented by the sum of the vectors for the central concepts (VL1+ . .. +VLh) which may be normalized by dividing the sum by its length. Asdisclosed herein, the vector sorting at step 604 may be performed at theQA system 100 or concept processing application 14 by employing anydesired sorting technique to sort the computed cosine distance from highto low, such as by sorting the vectors V′k+1, . . . , V′k+b by theirhighest cosine distance to the normalized sum (VL1+ . . . +VLh). Theconcept processing step 604 may then generate a sorted vector list LSalong with a list LSC of the corresponding concepts, and then report thetop R1 (new) concepts from the list of the corresponding concepts LSC,where R1 is a parameter (e.g., 3). In this way, the reported R1 conceptsare potentially emerging disruptive technologies, or other concepts, toT where T is viewed as a conglomeration of technologies.

To provide additional details for an improved understanding of selectedembodiments of the present disclosure, reference is now made to FIG. 7which depicts a simplified flow chart 701-705 showing the logic forusing a temporal comparison of two concept vector states to identifyemerging concepts with strong affinity to concepts that are central to atechnology area of interest. The processing shown in FIG. 7 may beperformed in whole or in part by a cognitive system, such as the QAinformation handing system 15, QA system 100, or other natural languagequestion answering system which identifies and compares temporallyseparated sequences of concepts to carry out useful tasks in the domainof concepts and user- concept interaction, such as identifying emergingconcepts in a new concept sequence or in both new and old conceptsequences that have significantly strenghted in a technology area orfield of interest that may be defined by a single concept (e.g., one of:music, politics, film, physics) or by a number of concepts (e.g.,republican, candidate, presidency).

FIG. 7 processing commences at step 701 by capturing, retrieving, orotherwise obtaining at least two temporally separated input sequences,including an “old” sequence S1 and a “new” sequence S2 spaced apart by acertain amount of time. In selected embodiments, the input sequences S1,S2 may be retrieved from storage in a database, where S1 contains theconcept set {C1, . . . , Ck}, and where S2 contains the “old” set {C1, .. . , Ck} and the “new” concept set {N1, . . . , Nb}. The processing atstep 701 may be performed at the QA system 100 or other NLP questionanswering system, though any desired information processing system forprocessing questions and answers may be used.

Once the concept sequences S1, S2 are available, the concept sequencesmay be processed to compute or train, on S1, concept vectors V1, . . . ,Vk, for the concepts in the first input sequence S1 using known vectorembedding techniques at step 702. In addition, the concept sequences S1,S2 are further processed at step 703 to compute or train concept vectorsV′1, . . . , V′k, V′k+1, . . . V′k+b over S2. At steps 702, 703, Vi andVi′, 1≤i≤k refers to concepts in both S1 and S2, while V′i, i>k (e.g.,i=k+1 . . . k+b) refer to concepts only in S2. As disclosed herein, theprocessing at steps 702, 703 may be performed at the QA system 100 orconcept vector extractor 12 by employing known vector embeddingtechniques, such as machine learning and/or NLP techniques, to compute adistributed representation (vectors) of concepts which are trained onthe concepts from the first concept sequence S1 (step 702) and thesecond concept sequence S2 (step 703).

At step 704, a field vector V′F (and similarly, VF over S1) isidentified as corresponding to a field of interest represented by one ormore concept vectors. For example, a field vector V′F can be the sum ofa plurality of concept vectors, such asV′singer+V′rock_music+V′concert+V′you_tube_video which are respectivelyassociated with the corresponding concepts, singer, rock_music, concert,you_tube_video. As disclosed herein, the processing at step 704 may beperformed at the QA system 100 or concept vector processing application14 by employing known vector processing techniques to construct thefield vector V′F as a normalized vector which is computed by dividingthe field vector V′F by its length. At this point in the process, it isobserved that the field vector V′F (and each of its constituents, ifany) is trained over S1S2.

At step 705, emerging concept vectors V′j, 1≤j≤k+b, are identified orlocated which meet three conditions or requirements. Under the firstrequirement, the number of occurrences of the concept Cj of V′j in thenew concept sequence S2 exceeds a threshold parameter value Th (e.g.,500). Under the second requirement, a computed cosine distance value,Aj=cos(V′j, V′F), must exceed a reporting threshold or parameter valueDELTA4 (e.g., DELTA=0.75). Under the third requirement, either j>k(indicating a new concept) or the computed cosine distance valueBj=cos(Vj, VF)+DELTA5 is less than Aj, where DELTA5 is a parameter (e.g.0.1). At this point in the process, it is observed that Vj is trainedover the first concept sequence S1. So, we are comparing the ‘strength’of the relationship between Vj and VF to that of V′j and V′F. Atprocessing step 705, the first requirement effectively detects asignificant presence, the second requirement enforces closeness to thefield of interest, and the third requirement ensures that there is asignificant change with respect to S1. As disclosed herein, theprocessing at step 705 may be performed at the QA system 100 or conceptvector application 14 by employing known vector processing techniques.Although the method was designed to identify emerging (i.e., increasingin importance) concepts, by changing DELTA5 to a negative quantity,diminishing concepts (i.e., decreasing in importance) can be identified.

By now, it will be appreciated that there is disclosed herein a system,method, apparatus, and computer program product for analyzing conceptvectors over time to detect changes in a corpus with an informationhandling system having a processor and a memory. As disclosed, thesystem, method, apparatus, and computer program product generate firstand second concept vector sets, respectively, from first and second setsof concept sequences applied to a vector learning component, where thesecond set of concept sequences is effectively collected aftercollection of the first set of concept sequences. The concept vectorsets are processed by performing a natural language processing (NLP)analysis of the first concept vector set and second concept vector setto detect vector changes in the corpus over time by identifying changesfor one or more concepts included in the first and/or second set ofconcept sequences. In selected embodiments, the NLP analysis includesanalyzing relationship strengths between concepts that persist in thefirst set of concept sequences and the second set of concept sequences.Such relationship strength analysis may include computing a first cosinedistance between each vector pair Vi, Vj from a first set of conceptvectors V1, . . . , Vk derived from the first set of concept sequencesover k “old” concepts (for all i≠j, 1≤i, j≤k) and also computing asecond cosine distance between each vector pair V′i, V′j from a secondset of concept vectors V′1, . . . , V′k+b derived from a concatenationof the first set of concept sequences over k “old” concepts and a secondset of concept sequences over k “old” and b “new” concepts (for all i≠j,1≤i,j≤k) thereby identifying concept pairs from the first set of conceptsequences whose interrelationship has changed by reporting each conceptpair Ci, Cj whereby a subtraction of the second cosine distance from thefirst cosine distance exceeds a first specified reporting threshold.(Note—from now on we will drop the quotation marks around “old” and“new”.) Alternatively, such relationship strength analysis may includecomputing a first cosine distance between each vector pair Vi, Vj from afirst set of concept vectors V1, . . . , Vk derived from the first setof concept sequences over k concepts (for all i≠j, i,j≤k) and alsocomputing a second cosine distance between each vector pair V′i, V′jfrom a second set of concept vectors V′1, . . . , V′k+b derived from asecond set of concept sequences over k old and b new concepts (for alli≠j, i,j≤k), thereby identifying concept pairs from the first set ofconcept sequences whose interrelationship has changed by reporting eachconcept pair Vi, Vj whereby a subtraction of the second cosine distancefrom the first cosine distance exceeds a first specified reportingthreshold. In other embodiments, the NLP analysis may include detectingan emergence of one or more new concepts in the second set of conceptsequences that are not present in the first set of concept sequences.The process for detecting the emergence of one or more new concepts mayinclude computing a first cosine distance between each vector pair V′i,V′j from a first set of concept vectors V′1, . . . V′k, V′k+1, . . . ,V′k+b derived from a concatenation of the first set of concept sequencesover k concepts and a second set of concept sequences over k old and bnew concepts (for 1<i≤k and k<j≤k+b), thereby identifying new conceptpairs from the second set of concept sequences over k old and b newconcepts having a strong interrelationship with concepts in the firstset of concept sequences by reporting each concept pair Vi, Vj wherebythe first cosine distance exceeds a first specified reporting threshold.In yet other embodiments, the NLP analysis may include detecting adisappearance of one or more old concepts from the first set of conceptsequences that are not present in the second set of concept sequences.In yet other embodiments, the NLP analysis may include detecting anappearance of one or more disruptive concepts in the second set ofconcept sequences that are related to a specified technology arearepresented by a sum of a plurality of concept vectors. In yet otherembodiments, the NLP analysis may include detecting an appearance of oneor more emerging concepts in the second set of concept sequences thatare related to a specified technology area.

While particular embodiments of the present invention have been shownand described, it will be obvious to those skilled in the art that,based upon the teachings herein, changes and modifications may be madewithout departing from this invention and its broader aspects.Therefore, the appended claims are to encompass within their scope allsuch changes and modifications as are within the true spirit and scopeof this invention. Furthermore, it is to be understood that theinvention is solely defined by the appended claims. It will beunderstood by those with skill in the art that if a specific number ofan introduced claim element is intended, such intent will be explicitlyrecited in the claim, and in the absence of such recitation no suchlimitation is present. For non-limiting example, as an aid tounderstanding, the following appended claims contain usage of theintroductory phrases “at least one” and “one or more” to introduce claimelements. However, the use of such phrases should not be construed toimply that the introduction of a claim element by the indefinitearticles “a” or “an” limits any particular claim containing suchintroduced claim element to inventions containing only one such element,even when the same claim includes the introductory phrases “one or more”or “at least one” and indefinite articles such as “a” or “an”; the sameholds true for the use in the claims of definite articles.

1-24. (canceled)
 25. A method, in an information handling systemcomprising a processor and a memory, for analyzing concept vectors todetect changes in a corpus over time, the method comprising: analyzingrelationship strengths between concepts that persist in a first set ofconcept sequences and a second set of concept sequences to identifymarket trends for answering questions submitted to the informationhandling system by identifying vector changes for one or more conceptsincluded in the first and/or second set of concept sequences, whereinanalyzing relationship strengths comprises: computing, by the system, afirst cosine distance between each vector pair Vi, Vj from a firstconcept vector set V1, . . . , Vk derived from the first set of conceptsequences over k concepts for all i≠j, 1≤i, j≤k; computing, by thesystem, a second cosine distance between each vector pair V′i, V′j froma second concept vector set V′1, . . . , V′k+b derived from the secondset of concept sequences over k old and b new concepts for all i≠j, 1≤i,j≤k; and identifying concept pairs from the first set of conceptsequences whose interrelationship has changed by reporting each conceptpair Vi, Vj whereby a subtraction of the second cosine distance from thefirst cosine distance exceeds a first specified reporting threshold. 26.The method of claim 25, further comprising generating, by the system,the first concept vector set V1, . . . , Vk that is derived from thefirst set of concept sequences over k concepts that are extracted fromthe corpus and applied to a vector learning component.
 27. The method ofclaim 25, further generating, by the system, the second concept vectorset V′1, . . . , V′k+b that is derived from the second set of conceptsequences over k old and b new concepts that are extracted from thecorpus and applied to the vector learning component, where the secondset of concept sequences is effectively collected after collection ofthe first set of concept sequences.
 28. The method of claim 25, whereinanalyzing relationship strengths comprises performing natural languageprocessing (NLP) analysis of the first and second concept vector sets todetect an appearance of one or more new concepts in the second set ofconcept sequences that are not present in the first set of conceptsequences.
 29. The method of claim 28, wherein detecting the appearanceof one or more new concepts comprises: computing, by the system, a firstthird cosine distance between each vector pair V′i, V′j from a thirdconcept vector set a first set of concept vectors V′1, . . . V′k, V′k+1,. . . , V′k+b derived from a concatenation of the first set of conceptsequences over k concepts and a second set of concept sequences over kold and b new concepts for 1<i<k and k<j≤k+b; and identifying newconcept pairs from the second set of concept sequences over k old and bnew concepts having a strong interrelationship with concepts in thefirst set of concept sequences by reporting each concept pair Vi, Vjwhereby the third first cosine distance exceeds a first specifiedreporting threshold.
 30. The method of claim 25, wherein analyzingrelationship strengths comprises performing natural language processing(NLP) analysis of the first and second concept vector sets to detect adisappearance of one or more old concepts from the first set of conceptsequences that are not present in the second set of concept sequences.31. The method of claim 25, wherein analyzing relationship strengthscomprises performing natural language processing (NLP) analysis of thefirst and second concept vector sets to detect an appearance of one ormore disruptive concepts in the second set of concept sequences that arerelated to a specified technology area represented by a sum of aplurality of concept vectors.
 32. The method of claim 25, whereinanalyzing relationship strengths comprises performing natural languageprocessing (NLP) analysis of the first and second concept vector sets todetect an appearance of one or more emerging concepts in the second setof concept sequences that are related to a specified topic area.
 33. Themethod of claim 25, wherein analyzing relationship strengths comprisesperforming natural language processing (NLP) analysis of the first andsecond concept vector sets to detect differences in spatial and/orfrequency distributions of first and second concept vector sets byidentifying changes in values of quantitative geometry and topologyfeatures that characterize concept regions associated, respectively,with the first and second concept vector sets.
 34. The method of claim33, wherein identifying changes comprises computing differences incentroid positions, diameters between extreme points, orientations ofthe principal axes, number of significant dimensions, aspect ratiosbetween lengths of the principal axes, or number and sizes of clusterscomputed by standard clustering algorithms.
 35. An information handlingsystem comprising: one or more processors; a memory coupled to at leastone of the processors; a set of instructions stored in the memory andexecuted by at least one of the processors to analyze concept vectors todetect changes in a corpus over time, wherein the set of instructionsare executable to perform actions of: analyzing, by the system,relationship strengths between concepts that persist in a first set ofconcept sequences and a second set of concept sequences to identifymarket trends for answering questions submitted to the informationhandling system by identifying vector changes for one or more conceptsincluded in the first and/or second set of concept sequences, whereinanalyzing relationship strengths comprises: computing, by the system, afirst cosine distance between each vector pair Vi, Vj from a firstconcept vector set V1, . . . , Vk derived from the first set of conceptsequences over k concepts for all i≠j, 1≤i, j≤k; computing, by thesystem, a second cosine distance between each vector pair V′i, V′j froma second concept vector set V′1, . . . , V′k+b derived from the secondset of concept sequences over k old and b new concepts for all i≠j, 1≤i,j≤k; and identifying concept pairs from the first set of conceptsequences whose interrelationship has changed by reporting each conceptpair Vi, Vj whereby a subtraction of the second cosine distance from thefirst cosine distance exceeds a first specified reporting threshold. 36.The information handling system of claim 35, wherein the set ofinstructions are executable to generate the first concept vector set V1,. . . , Vk that is derived from the first set of concept sequences overk concepts that are extracted from the corpus and applied to a vectorlearning component.
 37. The information handling system of claim 35,wherein the set of instructions are executable to generate the secondconcept vector set V′1, . . . , V′k+b that is derived from the secondset of concept sequences over k old and b new concepts that areextracted from the corpus and applied to the vector learning component,where the second set of concept sequences is effectively collected aftercollection of the first set of concept sequences.
 38. The informationhandling system of claim 35, wherein the set of instructions areexecutable to analyze relationship strengths by performing naturallanguage processing (NLP) analysis of the first and second conceptvector sets to detect an appearance of one or more new concepts in thesecond set of concept sequences that are not present in the first set ofconcept sequences.
 39. The information handling system of claim 38,wherein the set of instructions are executable to detect the appearanceof one or more new concepts by: computing, by the system, a first thirdcosine distance between each vector pair V′i, V′j from a third conceptvector set a first set of concept vectors V′1, . . . V′k, V′k+1, . . . ,V′k+b derived from a concatenation of the first set of concept sequencesover k concepts and a second set of concept sequences over k old and bnew concepts for 1<i<k and k<j≤k+b; and identifying new concept pairsfrom the second set of concept sequences over k old and b new conceptshaving a strong interrelationship with concepts in the first set ofconcept sequences by reporting each concept pair Vi, Vj whereby thethird first cosine distance exceeds a first specified reportingthreshold.
 40. A computer program product stored in a non-transitorycomputer readable storage medium, comprising computer instructions that,when executed by an information handling system, causes the system toanalyze concept vectors to detect changes in a corpus over time byperforming actions comprising: analyzing, by the system, relationshipstrengths between concepts that persist in a first set of conceptsequences and a second set of concept sequences to identify markettrends for answering questions submitted to the information handlingsystem by identifying vector changes for one or more concepts includedin the first and/or second set of concept sequences, wherein analyzingrelationship strengths comprises: computing, by the system, a firstcosine distance between each vector pair Vi, Vj from a first conceptvector set V1, . . . , Vk derived from the first set of conceptsequences over k concepts for all i≠j, 1≤i, j≤k; computing, by thesystem, a second cosine distance between each vector pair V′i, V′j froma second concept vector set V′1, . . . , V′k+b derived from the secondset of concept sequences over k old and b new concepts for all i≠j, 1≤i,j≤k; and identifying concept pairs from the first set of conceptsequences whose interrelationship has changed by reporting each conceptpair Vi, Vj whereby a subtraction of the second cosine distance from thefirst cosine distance exceeds a first specified reporting threshold. 41.The computer program product of claim 40, wherein the computerinstructions are executed to generate the first concept vector set V1, .. . , Vk that is derived from the first set of concept sequences over kconcepts that are extracted from the corpus and applied to a vectorlearning component.
 42. The information handling system of claim 35,wherein the computer instructions are executed to generate the secondconcept vector set V′1, . . . , V′k+b that is derived from the secondset of concept sequences over k old and b new concepts that areextracted from the corpus and applied to the vector learning component,where the second set of concept sequences is effectively collected aftercollection of the first set of concept sequences.
 43. The informationhandling system of claim 35, wherein analyzing relationship strengthscomprises performing natural language processing (NLP) analysis of thefirst and second concept vector sets to detect an appearance of one ormore new concepts in the second set of concept sequences that are notpresent in the first set of concept sequences.
 44. The informationhandling system of claim 43, wherein detecting the appearance of one ormore new concepts comprises: computing, by the system, a first thirdcosine distance between each vector pair V′i, V′j from a third conceptvector set a first set of concept vectors V′1, . . . V′k, V′k+1, . . . ,V′k+b derived from a concatenation of the first set of concept sequencesover k concepts and a second set of concept sequences over k old and bnew concepts for 1<i<k and k<j≤k+b; and identifying new concept pairsfrom the second set of concept sequences over k old and b new conceptshaving a strong interrelationship with concepts in the first set ofconcept sequences by reporting each concept pair Vi, Vj whereby thethird first cosine distance exceeds a first specified reportingthreshold.